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1.  Introduction 


U.S.  Army  dismounted  Soldiers  perform  tasks  that  require  physical  and  cognitive  resources.  For 
example,  dismounted  Soldiers  often  attend  to  information  from  radio  networks  and  communicate 
with  members  of  their  squad  (a  task  involving  cognitive  skills)  while  simultaneously  using  a  map 
to  navigate  over  unfamiliar  terrain  (a  task  involving  cognitive  and  perceptual/motor  skills). 

While  performing  these  tasks,  Soldiers  must  also  be  capable  of  moving  their  body  mass  plus  an 
external  load  during  high-speed,  potentially  prolonged  tactical  operations  (Mahoney  et  al.,  2007). 

Technologies  have  been  developed  to  assist  the  dismounted  Soldier  in  performing  his/her 
mission.  Handheld  communication  and  navigation  devices  have  been  developed  to  provide 
dismounted  Soldiers  with  additional  capabilities  to  enhance  their  knowledge  of  the  battlefield. 

For  example,  the  Rifleman  Radio  (AN/PRC- 154)  was  designed  to  allow  voice  and  data 
communication  as  well  as  tracking  of  individual  Soldier  GPS  locations  for  navigation  (Bertucca, 
2012;  DOD  Programs,  2011;  General  Dynamics,  2013).  However,  when  using  these  systems,  the 
effect  of  physical  load  and  the  presence  of  other  battlefield  tasks  (e.g.,  monitoring  radio 
messages  while  navigating  across  the  battlefield)  will  significantly  affect  how  successful  the 
Soldier  is  at  detecting,  processing,  and  responding  to  important  battlefield  information. 

Technologies  such  as  virtual  environments  have  been  developed  to  allow  dismounted  Soldiers  to 
train  while  mitigating  risk.  One  such  immersive  virtual  reality  system,  the  Dismounted  Soldier 
Training  System  at  Fort  Bragg,  NC  (Bymer,  2012),  was  designed  to  provide  dismounted  Soldiers 
with  training  in  several  different  battlefield  conditions,  while  allowing  the  performance  of 
navigation  and  radio  communication  tasks  in  real  time.  Although  virtual  simulations  exist  to  train 
Soldiers  for  battlefield  tasks,  little  research  exists  that  compares  Soldier  performance  in  both  real 
and  virtual  (simulated)  battlefield  tasks. 

Few  researchers  have  tested  the  effectiveness  of  Army-relevant  auditory  vigilance  tasks  such  as 
radio  check  tasks  as  tools  for  measuring  dismounted  Soldier  cognitive  performance.  Although 
Crowell  et  al.,  (in  press)  used  a  radio  check  task,  this  task  was  relatively  undemanding  as  a 
cognitive  stressor.  During  the  Crowell  et  al.  radio  check  task,  the  Soldier  participants  did  not 
have  any  other  cognitive  tasks  to  attend  to,  and  the  only  messages  on  the  “radio  network”  were 
the  radio  check  messages. 

The  current  experiment  investigated  three  issues:  (1)  the  effect  of  different  physical  loads  on 
Soldier  performance  of  navigation  and  radio  check  tasks,  (2)  the  effect  of  real  and  virtual 
environments  on  Soldier  performance  of  the  same  tasks,  and  (3)  a  more  demanding  radio  check 
task  to  determine  whether  radio  check  response  time,  duration,  and  accuracy  had  potential  as 
measures  of  Soldier  cognitive  performance. 
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1.1  The  Effect  of  Physical  Load  on  Soldier  Performance 

There  is  little  agreement  regarding  how  physical  load  or  physical  exertion  affects  performance  in 
navigation  and  radio  check  tasks.  Some  researchers  found  a  beneficial  relationship  between  load 
carriage  and  cognitive  performance.  Hogervorst  et  al.  (1996)  found  improved  performance  on 
cognitive  tasks  after  exercise  on  a  cycle  ergometer.  Lybrand  et  al.  (1954)  assessed  the  effects  of  a 
5-mile  march  with  a  40-lb  pack  on  the  perceptual  ability  of  college  students,  finding  that  scores 
on  perceptual  tasks  were  higher  after  mild  physical  activity  than  during  periods  of  no  exercise 
and  sleep  deprivation.  Gliner  et  al.  (1979)  observed  that  aerobic  energy  produced  in  a  marathon 
race  facilitated  the  performance  of  adult  men  in  a  vigilance  signal-detection  task. 

Some  researchers  found  a  detrimental  relationship  between  load  carriage  and  cognitive 
performance.  Mahoney  et  al.  (2007)  found  that  walking  affects  concurrent  cognitive  performance 
in  a  secondary  auditory  vigilance  task,  and  that  there  were  fewer  correct  vigilance  task  responses 
when  carrying  a  load  or  when  walking  over  obstacles  compared  with  standing  still  with  and 
without  a  load.  Knapik  et  al.  (1990)  found  significant  decrements  in  military  performance  after 
Soldiers  had  completed  a  20-km  road  walk  while  carrying  a  total  of  101  lb.  This  load  not  only 
made  the  road  march  extremely  strenuous,  but  led  to  inhibited  grenade  throw  performance  and 
decreased  marksmanship  accuracy.  In  addition,  after  the  road  march,  the  Soldiers  reported  a 
significant  decline  in  vigor  and  increase  in  fatigue.  Drain  et  al.  (2010)  noted  that  prolonged  load 
carriage  may  deplete  or  diminish  cognitive  performance,  but  that  with  high  physical  demands 
there  were  no  changes  in  cognitive  performance  on  a  complex  synthetic  work  task  conducted 
after  the  completion  of  a  road  march  task.  Kobus  et  al.  (201 1)  found  that  cognitive  performance 
degraded  with  increasing  load  carried,  physiological  strain  and  cognitive  performance  were 
negatively  correlated,  and  cognitive  performance  was  positively  correlated  with  perceived 
comfort  and  negatively  correlated  with  perceived  heaviness. 

Some  researchers  found  mixed  results  on  the  effects  of  load  on  cognitive  ability.  Knapik  et  al. 
(1997)  found  that  strenuous  load  carriage  (34-61  kg  carried  20  km)  led  to  decrements  in 
subsequent  physical  performance  but  not  in  cognitive  ability.  Crowell  et  al.  (1999)  found  that 
error  rates  in  an  auditory  monitoring  cognitive  performance  task  were  significantly  lower  for  a 
light  load  (22.77  kg)  than  for  a  heavier  load  (36.94  kg).  They  also  found  that  there  were  no 
significant  differences  in  performance  on  arithmetic  and  memory  tasks  as  a  function  of  load. 

This  may  have  been  due  to  the  structured  presentation  of  the  questions  or  because  the  physical 
exertion  was  not  intense  enough  to  affect  task  performance. 
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Mahoney  et  al.  (2007)  hypothesized  that  the  lack  of  agreement  between  research  results  may  be 
due  to  differences  in  the  nature,  duration,  and  intensity  of  the  physical  exertion  of  the 
experimental  participants,  the  nature  of  the  cognitive  tasks  used,  and  the  fitness  level  of  the 
participants.  Other  differences  may  have  been  due  to  some  researchers  obtaining  data  after  rather 
than  during  exercise  and  load-bearing  activities. 

The  load  carriage  aspect  of  the  present  study  explored  whether  a  relatively  “light”  load  (24-kg 
fighting  load  in  a  rucksack)  carried  in  a  warehouse  environment  would  have  an  effect  on  Soldier 
performance  in  a  navigation  task  (requiring  cognitive  and  perceptual/motor  skills)  and  in  an 
auditory  radio  check  task  (requiring  cognitive  skills).  Performance  on  these  two  tasks  while 
carrying  the  24-kg  load  was  compared  with  no  load  carried  while  navigating  around  the  same 
warehouses,  as  well  as  in  two  virtual  environments — virtual  warehouses  shown  in  a  cave 
automatic  virtual  environment  (CAVE)  and  virtual  warehouses  shown  on  a  computer  monitor. 

To  decrease  the  problems  noted  by  Mahoney  et  al.  (2007),  the  fitness  level  of  participants  was 
similar;  all  participants  were  Soldiers,  National  Guard,  or  recent  Army  retirees,  and  all  measures 
were  taken  during  task  activities.  Based  upon  results  of  Kobus  et  al.  (201 1),  it  was  hypothesized 
that  performance  in  the  navigation  and  the  radio  check  tasks  would  be  negatively  correlated  with 
load  weight. 

1.2  The  Effect  of  Real  and  Virtual  Conditions  on  Soldier  Performance 

Although  virtual  simulations  exist  to  train  Soldiers  for  battlefield  tasks,  little  research  has 
explored  Soldier  cognitive  performance  under  comparable  real  and  virtual  environments.  Witmer 
et  al.  (1996)  found  differences  in  the  number  of  wrong  turns,  but  not  speed  or  route  completion, 
when  comparing  participant  performance  while  traveling  a  specific  route  within  a  real  and  a 
virtual  building.  These  researchers  found  that  participants  trained  in  the  virtual  environment 
made  more  wrong  turns  than  those  trained  in  the  real-world  environment.  In  addition,  Witmer 
and  Kline  (1998)  found  that  tasks  such  as  distance  estimation  are  less  accurate  in  a  virtual 
environment  than  in  the  real  world.  Klein  (1976)  suggested  that  “moving  per  se  does  not  always 
require  attention;  rather,  it  is  the  guidance  of  movement,  in  terms  of  direction  and  speed  that 
requires  attention.”  This  would  imply  that  guidance  of  movement  in  virtual  worlds  (e.g.,  with  a 
joystick  guiding  direction  and  speed)  demands  more  cognitive  work  than  navigating  a  real-world 
environment. 

In  order  to  investigate  Klein’s  (1976)  suggestion  that  guidance  of  movement  in  virtual  worlds 
demands  more  cognitive  work  than  walking  in  a  real  world,  this  study  partially  replicated  the 
methods  used  in  Crowell  et  al.  (in  press).  In  the  latter  study,  each  participant  performed  a  map 
navigation  task  in  and  around  the  U.S.  Army  Research  Laboratory  (ARL)  warehouse  complex 
without  a  physical  load.  In  addition,  Soldiers  wearing  no  load  used  a  joystick  to  navigate  through 
a  digital  (virtual)  representation  of  the  ARL  warehouse  complex  as  experienced  on  a  desktop 
computer  and  as  experienced  in  the  ARL  CAVE  environment.  In  their  study,  Crowell  et  al.  had 
the  Soldier  walk  on  an  Omni  Directional  Treadmill  within  the  CAVE,  but  noise  issues  prevented 
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complete  analysis  of  the  data  from  the  radio  task  in  that  environment.  Based  on  Klein  (1976),  it 
was  hypothesized  that  navigation  in  virtual  environments  (following  a  course  in  CAVE  and 
computer  monitor  environments)  would  require  more  attention  (and  thus  more  time)  than 
navigation  through  real-world  environments  (following  a  course  around  the  ARL  warehouse 
with  and  without  a  24-kg  load). 

1.3  The  Potential  of  Radio  Check  Response  Time,  Duration,  and  Accuracy  as  Measures  of 
Dismounted  Soldier  Cognitive  Performance 

To  study  the  effects  of  Soldier  cognitive  performance,  Crowell  et  al.  (in  press)  used  a  radio  check 
auditory  vigilance  task  as  a  secondary  task  cognitive  stressor,  having  Soldiers  respond  to  targeted 
auditory  radio  check  requests  to  engage  Soldiers’  spare  mental  capacity,  while  simultaneously 
performing  a  walking  task  on  the  ARL  Ground  Vehicle  Experimentation  Course  at  different  set 
speeds.  Pairing  an  auditory  task  with  a  simultaneous  visual  navigation  task  was  thought  to  be 
advantageous;  Wickens’s  (1984)  multiple  resource  theory  stated  that  different  tasks  can  be 
processed  in  parallel  if  the  tasks  required  different  resource  modalities.  Scherer  et  al.  (2002) 
suggested  that  auditory  cognitive  and  attentional  demands  affect  speech  fluency  and  speech  rate, 
and  that  measures  used  in  the  radio  check  task  (response  time,  duration,  and  accuracy)  are  all 
measures  of  speech  fluency  and  speech  rate.  Thus,  by  using  the  auditory  modality  for  input  and 
speech  for  output,  it  was  thought  that  an  auditory  radio  check  vigilance  task  would  not  interfere 
with  the  navigation  task’s  visual  input  and  motor  output. 

In  the  present  study,  the  radio  check  task  was  made  more  demanding  than  that  used  in  Crowell 
et  al.  by  inserting  redesigned  radio  check  messages  within  natural  pauses  in  an  otherwise 
continuous  stream  of  simulated  radio  traffic.  The  radio  traffic  consisted  of  scripted  messages 
spoken  by  actors  posing  as  Soldiers  performing  an  Army-relevant  dismounted  Soldier  mission. 

In  addition,  a  larger  number  of  target  and  nontarget  radio  check  messages  were  used,  each  of 
which  required  immediate  Soldier  response.  It  was  thought  that  because  the  radio  check  task  was 
embedded  within  almost  constant  radio  traffic,  Soldier  attention  would  be  engaged  to  a  high 
degree,  especially  when  used  in  tandem  with  the  navigation  task.  It  was  hypothesized  that 
Soldier  response  time  and  accuracy  in  the  radio  check  task  could  provide  an  effective  tool  to 
measure  dismounted  Soldier  cognitive  performance.  In  the  future,  this  redesigned  radio  check 
task  could  provide  an  effective  tool  to  measure  dismounted  Soldier  cognitive  performance. 
Measures  used  in  the  radio  check  task  (response  time,  duration,  and  accuracy)  are  all  measures  of 
speech  fluency  and  speech  rate  (Scherer  et  al.,  2002).  Because  of  the  relationship  between 
cognitive  performance  and  radio  check  task  speech  measures,  it  was  hypothesized  that  radio 
check  task  response  time,  duration,  and  accuracy  would  have  the  potential  for  reflecting 
meaningful  differences  in  Soldier  performance. 
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2.  Research  Objective 


The  objective  of  this  study  was  to  develop  a  characterization  of  the  interaction  of  physical  load 
and  environment  on  dismounted  Soldier  cognitive  performance  by  exploring  the  effect  of 
physical  load  on  Soldier  performance,  the  effect  of  real  and  virtual  environments  on  Soldier 
performance,  and  the  potential  of  radio  check  response  time,  duration,  and  accuracy  as  measures 
of  dismounted  Soldier  cognitive  performance.  The  resolution  of  these  issues  would  allow 
researchers  to  choose  the  most  appropriate  and  relevant  environments  and  tasks  for  the  study  of 
physical  and  cognitive  stressors  on  dismounted  Soldiers.  In  addition,  data  from  this  study  could 
be  used  to  inform  system  design,  mission  load  tradeoffs,  and  distribution  of  tasks,  as  well  as 
provide  input  to  modeling  and  simulation  tools. 

2.1  Hypotheses 

2.1.1  The  Effect  of  Physical  Load  on  Soldier  Performance 

It  was  hypothesized  that  Soldier  performance  on  the  navigation  and  radio  check  tasks  would  be 
negatively  correlated  with  load  weight.  It  was  also  hypothesized  that  navigation  and  radio 
response  performance  time  means  for  the  24-kg  load  environment  would  be  significantly  greater 
than  the  means  for  the  no-load  environments. 

2.1.2  The  Effect  of  Real  and  Virtual  Environments  on  Soldier  Performance 

It  was  hypothesized  that  navigation  in  virtual  environments  (CAVE  and  computer  monitor) 
would  require  more  time  and  attention  than  navigation  in  real-world  environments  (Warehouse 
With  Load  and  Warehouse  Without  Load).  Thus,  Soldier  navigation  and  radio  check  mean 
performance  for  the  virtual  environments  would  be  significantly  greater  than  mean  performance 
for  the  real-world  environments 

2.1.3  The  Potential  of  Radio  Check  Response  Time,  Duration,  and  Accuracy  as  Measures  of 
Dismounted  Soldier  Cognitive  Performance 

Radio  check  tasks  employ  cognitive  processing.  Because  of  the  relationship  between  cognitive 
performance  and  speech  measures  used  in  the  radio  check  task,  it  was  hypothesized  that  radio 
check  task  response  time,  duration,  and  accuracy  would  reflect  meaningful  differences  in  Soldier 
cognitive  performance  in  the  different  environments  used  in  this  study. 
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3.  Method 


3.1  Participants 

There  were  16  participants  in  this  study,  all  of  whom  were  active  duty  U.S.  Soldiers,  National 
Guard,  or  recent  Army  retirees.  There  were  15  males  and  1  female  between  the  ages  of  22  and  40 
(mean  [M]  =  28.54  years,  standard  deviation  [SD]  =  5.91).  Fourteen  participants  (13  males  and  1 
female)  were  Soldiers  from  the  22d  Chemical  Battalion  at  Aberdeen  Proving  Ground  (APG), 
Edgewood,  MD.  Two  male  participants  (one  an  Army  National  Guard  Explosive  Ordnance 
Disposal  Specialist  and  one  a  recent  Army  retiree  currently  working  for  the  APG  Special 
Weapons  and  Tactics  Team)  were  recruited  from  a  local  temporary  employment  agency  and 
were  paid  $30  per  hour  for  their  participation  in  this  study.  Of  the  14  chemical  battalion  Soldiers 
from  APG,  two  had  the  rank  of  private  E-2,  seven  were  specialists,  one  was  a  corporal,  two  were 
sergeants,  one  was  a  staff  sergeant,  and  one  was  a  sergeant  first  class. 

All  participants  had  an  active  driver’s  license  and  had  self-reported  good  health  and  normal 
hearing  and  vision  (including  corrected  vision).  No  participant  had  any  illness  or  injury  that 
would  affect  his/her  ability  to  walk,  see,  or  hear.  The  voluntary,  fully  informed  consent  of  the 
persons  who  participated  in  this  research  was  obtained  by  U.S.  Army  human  use  regulations 
(U.S.  Department  of  Defense,  1999;  U.S.  Department  of  the  Army,  1990). 

3.2  Location 

This  study  was  conducted  at  two  locations.  All  warehouse  navigation  tasks  were  conducted  at  the 
ARL  Warehouse  Site,  Buildings  501-505,  at  APG.  The  Warehouse  Site  is  approximately  5  acres 
in  size,  relatively  flat,  and  has  roadways  paved  with  concrete,  blacktop,  or  packed  gravel.  All 
virtual  navigation  tasks  were  performed  at  the  ARL  Tactical  Environment  Simulation  Facility 
(TESF),  Building  518,  APG. 

3.3  Apparatus 

The  experimental  apparatus  consisted  of  two  simulation  systems,  a  joystick,  a  U.S.  Army 
rucksack,  and  hardware  and  software  for  the  radio  check  task. 

3.3.1  Simulation  Systems 

Two  different  TESF  simulation  systems  were  used  to  present  the  virtual  warehouse  environment 
to  the  participant.  Each  of  the  two  systems  showed  the  same  software,  which  contained  a 
digitized  version  of  the  ARL/ APG  Warehouse  Site,  Buildings  501-505,  including  all  paths  and 
roadways. 
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The  first  simulation  system  (a  computer  with  a  monitor)  consisted  of  one  Inter-based  Dell  ' 
computer  with  an  Intel  motherboard  and  a  256-MB  video  card  that  controlled  the  24-in.  flat 
panel  computer  monitor,  which  showed  the  virtual  warehouse  environment  to  the  participant. 
The  participant  sat  on  a  chair  at  a  table  with  the  monitor  placed  approximately  1 .5  ft  away  from, 
and  directly  in  front  of,  the  participant  (see  figure  1). 


Figure  1.  Simulation  system  with  computer  monitor. 

The  second  system  was  the  CAVE,  a  projection-based  immersive  virtual  reality  system.  The 
participant  sat  at  a  small  table  surrounded  by  four  large  display  screens,  each  located  6.25  ft 
away  from  the  participant  (see  figure  2).  Each  display  screen  measured  12.5-  x  10-ft,  and  all 
were  positioned  at  90°  to  one  another  with  the  participant  in  the  center.  The  screens  provided  the 
participant  with  a  full  360°  field  of  view  of  the  terrain,  providing  the  participant  the  experience 
of  immersion  in  the  virtual  environment.  The  terrain  images,  generated  by  a  cluster  of  four  Intel- 
based  Quantum  3-D,  Inc.,  computers  with  256-MB  video  cards,  were  rear-projected  onto  the 
CAVE  screens  by  four  projectors,  each  with  a  brightness  rating  of  6000  lm  (Christie  Digital 
Mirage  6000*). 


Intel  is  a  registered  trademark  of  Intel  Corporation,  Santa  Clara,  CA. 

’  Dell  is  a  registered  trademark  of  Dell  Incorporated,  Round  Rock,  TX. 

1  Christie  Digital  Mirage  6000  is  a  registered  trademark  of  Christie  Digital,  Cypress,  CA. 
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Figure  2.  Simulation  system  with  CAVE  display. 


3.3.2  Joystick 

A  Cyborg  3D  Rumble  Force  joystick  (Saitek,  Inc.,  San  Diego,  CA)  was  placed  on  the  top  of  the 
table  at  which  the  participant  sat  and  was  used  by  the  participant  to  control  movement  (direction 
and  rate  of  travel)  through  each  virtual  warehouse  environment.  The  maximum  speed  of  travel 
through  the  virtual  environment  was  limited  to  4  mph  (the  average  speed  of  travel  through  the 
real-world  warehouse  environment,  as  determined  by  Crowell  et  al.  [in  press]). 

3.3.3  U.S.  Army  Rucksack 

In  environments  in  which  participants  carried  a  fighting  load,  each  participant  wore  a  U.S.  Army 
rucksack  on  their  back  (see  figure  3).  A  foam  block  with  lead  shot  was  placed  inside  the 
rucksack.  The  total  weight  of  the  rucksack  with  the  foam  block  and  lead  shot  was  24  kg  (53  lb). 


Figure  3.  Rucksack  earned  for  fighting 
load. 


8 


3.3.4  Hardware  and  Software  for  the  Radio  Check  Task 

All  auditory  stimuli  (radio  traffic  and  radio  checks)  were  recorded  and  mixed  using 
Sound  Forge* * * §  version  4.5  software.  One  SanDisk  Sansa  Clip  Zip  MP3  Player1'  (1.0  x  1.0  in), 
connected  to  an  Auvio1  miniature  loudspeaker  (1.0  x  1.5  x  0.35  in),  was  used  to  present  all 
auditory  stimuli  to  the  participant.  The  auditory  stimuli  radio  messages  (Haas,  2012)  were  played 
at  68  dBA,  as  measured  at  the  level  of  the  participant’s  ear.  A  RipFlash  Pro  Digital  Audio  MP3 
Player/  Recorder^  (2.0  x  3.0  x  0.15  in)  was  used  to  record  all  participant  responses,  along  with 
radio  task  stimuli.  The  audio  hardware  was  attached  to  a  metal  audio  apparatus  fixture 
(2.0  x  4.0  in),  which  was  attached  to  the  front  strap  of  the  rucksack  in  the  trials  in  which  the 
rucksack  was  used.  The  fixture  was  attached  to  a  lightweight  harness  worn  by  the  participant  for 
all  conditions  in  which  the  rucksack  was  not  worn  (see  figure  4).  All  audio  equipment,  including 
the  fixture,  weighed  less  than  32  oz.  After  the  experiment,  Sound  Forge  version  4.5  and  Audacity 
version  1.3.12-beta  audio  software  were  used  to  analyze  the  radio  check  task  audio  data.  Dragon 
Dictate**  Speech  Recognition  software  version  5.0  was  then  used  to  write  the  audio  response 
time  data  into  an  Excel  file,  for  import  into  SPSS  software  for  the  statistical  analysis. 


a  b 

Figure  4.  Audio  hardware  attached  to  (a)  rucksack  and  (b)  harness. 


3.3.5  Questionnaires 

The  four  questionnaires  used  in  this  study  consisted  of:  (1)  the  participant  data  form,  a 
demographic  questionnaire  with  additional  questions  including  participant  health  status  and 
previous  computer  experience;  (2)  the  Santa  Barbara  Sense  of  Direction  Scale  (Hegarty  et  al., 


Sound  Forge  is  a  registered  trademark  of  Sony  Creative  Software,  Tokyo,  Japan. 

^  SanDisk  Sansa  Clip  Zip  MP3  Player  is  a  registered  trademark  of  SanDisk  Corporation,  Milpitas,  CA. 

1  Auvio  is  a  registered  trademark  of  RadioShack  Corporation,  Fort  Worth,  TX. 

§  RipFlash  Pro  Digital  Audio  MP3  Player/  Recorder  is  a  registered  trademark  of  PoGo!  Products,  Inc.,  Brea,  CA. 

Dragon  Dictate  is  a  registered  trademark  of  Nuance  Communications,  Inc.,  Burlington,  MA. 


9 


2002),  to  obtain  information  regarding  participant  self-reported  spatial  and  navigational  abilities; 
(3)  the  Perspective  Taking/Spatial  Orientation  Test  (Hegarty  and  Waller,  2004),  to  measure 
participant  ability  to  imagine  different  perspectives  or  orientations  in  space;  and  (4)  the  final 
questionnaire,  to  measure  the  participant’s  level  of  comfort,  physical  effort,  and  ease  of 
performance  during  both  the  audio  and  navigation  tasks. 

3.3.6  Other  Apparatus 

Reflective  safety  belts  were  worn  by  experimental  participants  and  test  administrators  when  they 
walked  upon  the  streets  and  roadways  of  the  ARL  Warehouse  Site. 

3.4  Stimuli 

3.4.1  Visual  (Navigation  Task)  Stimuli 

The  participants  used  a  map  to  navigate  a  specified  route  through  the  real  and  virtual  ARL 
Warehouse  Site  environments.  Four  different  maps  were  used  in  the  study  (one  map  for  each  of 
the  four  experimental  trials),  shown  in  figures  5-8.  One  additional  map  similar  to  those  shown  in 
figures  5-8  was  used  for  training  purposes.  Each  of  the  maps,  which  were  hand  drawn  on  two 
sides  of  8.5-  x  1 1-in  white  paper  and  laminated,  had  different  routes  through  the  ARL 
Warehouse  Site.  All  map  routes  covered  the  same  distance  but  on  each  map  the  direction  of 
travel  was  different  and  each  had  a  different  starting  point. 
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3.4.2  Audio  (Radio  Check  Task)  Stimuli 

An  MP3  file  of  simulated  U.S.  Army  radio  traffic  messages  (Mermagen,  1992)  was  used  to 
produce  five  different  MP3  audio  recordings  (one  for  each  of  the  four  experimental  trials  and  one 
for  training)  (Haas,  2012).  The  radio  traffic  recording  included  messages  such  as  situation 
reports,  line  of  advance  messages,  and  instructions  for  squad  members  to  move  to  other 
locations.  For  each  of  the  five  recordings,  30  different  prerecorded  target  radio  checks  spoken  by 
a  male  voice  were  presented  in  natural  pauses  in  the  radio  traffic.  Ten  of  the  radio  check 
messages  were  target  radio  checks  in  which  the  male  voice  spoke  the  call  sign  of  the  participant 
(“x-ray  two-three”),  identified  himself,  and  requested  a  radio  check  (“x-ray  two-three,  this  is  golf 
five-six,  radio  check,  over”).  This  message  cued  the  participant  to  reply  with  a  specific  phrase 
(“golf  five-six,  this  is  x-ray  two-three,  roger,  out”).  The  remaining  20  messages  were  nontarget 
radio  checks  in  which  the  same  male  voice  requested  a  radio  check  from  five  different  call  signs 
other  than  “x-ray  two-three”  (e.g.,  “alpha  two-five,  this  is  golf  five-six,  radio  check,  over”).  Each 
of  the  five  nontarget  call  signs  was  repeated  four  times  on  each  MP3  file.  The  participant  was 
told  not  to  reply  to  messages  not  targeted  to  him/her.  Fifteen  seconds  of  radio  silence  followed 
each  of  the  target  and  nontarget  radio  check  messages,  providing  time  for  the  participant  to 
respond  if  they  chose.  All  five  MP3  files  had  a  total  duration  of  approximately  14  min.  All  10 
target  and  20  nontarget  radio  check  messages  were  played  within  the  first  8  min  of  each  14-min 
audio  recording  to  ensure  that  the  participant  was  exposed  to  all  radio  checks  before  he/she 
finished  the  navigation  task.  The  navigation  task  was  designed  to  take  approximately  1 1  min  to 
perform  (as  per  Crowell  et  al.,  in  press). 

3.5  Procedure 

Only  one  person  participated  per  day,  and  for  each  participant,  all  environments  were  presented 
on  that  day.  The  participant  was  asked  if  he/she  had  normal  hearing  with  no  ear-related  problems 
or  infections.  If  the  participants  reported  ear  infections  or  if  they  did  not  meet  the  vision  criteria 
(reported  normal  20:30  vision  or  better  with  or  without  corrective  lenses),  they  were  not  allowed 
to  participate  in  the  study.  Participants  who  met  the  criteria  were  asked  to  read  and  sign  the 
informed  consent  form  (appendix  A)  and  assigned  a  number  to  identify  their  data. 

Next,  the  participant  provided  demographic  and  medical  status  information  on  the  participant 
data  form  (appendix  B)  and  completed  the  Santa  Barbara  Sense  of  Direction  Scale  and 
Perspective  Taking/Spatial  Orientation  Test  (appendixes  C  and  D). 

In  this  study,  Soldiers  used  handheld  paper  maps  to  navigate  through  four  different  environments 
while  listening  to  an  audio  recording  and  responding,  as  directed  by  the  experimenter,  to  target 
radio  checks.  First,  the  experimenter  trained  the  participant  to  perform  the  radio  check  task.  The 
participant  was  told  that  their  call  sign  would  be  “x-ray  two-three”  for  the  duration  of  the 
experiment.  The  participant  listened  to  an  example  target  radio  check  message,  was  told  the 
correct  response  to  that  message  (“[call  sign  of  caller],  this  is  x-ray  two-three,  roger,  out”),  and 
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repeated  that  response  to  the  experimenter.  The  participant  was  told  to  respond  only  to  radio 
check  messages  where  the  caller  used  the  participant’s  call  sign.  The  experimenter  then 
presented  examples  of  nontarget  radio  check  messages  (messages  that  did  not  use  the 
participant’s  call  sign)  to  the  participant  and  told  the  participant  not  to  respond  to  those 
messages.  Next,  the  participant  listened  and  responded  to  a  3-min  sample  of  simulated  radio 
traffic  that  contained  five  target  and  four  nontarget  radio  checks.  The  participant  was  given 
feedback  on  the  correctness  of  his  or  her  responses  by  the  experimenter.  After  the  participant 
made  five  correct  responses  to  five  sequential  target  radio  check  messages,  that  part  of  the 
training  was  complete. 

Next,  the  experimenter  familiarized  the  participant  with  the  warehouse  site,  driving  the 
participant  around  the  warehouse  area  while  he/she  used  the  training  map  to  locate  the 
warehouse  buildings  and  roads  used  in  the  navigation  task.  The  experimenter  also  described  the 
navigation  task  to  the  participant.  During  the  drive,  the  experimenter  answered  all  participant 
questions.  After  all  questions  were  answered,  this  phase  of  training  was  complete. 

The  participant  was  then  readied  for  the  first  navigation  trial  in  the  first  environment.  He/she 
donned  the  harness  or  rucksack  for  that  condition  along  with  the  fixture  containing  the  MP3 
recorder  and  player.  The  participant  was  then  handed  one  of  the  four  navigation  task  maps,  taken 
to  the  starting  position,  and  then  performed  an  experimental  trial  by  navigating  the  route  shown 
on  the  map  while  simultaneously  listening  to  the  audio  recording  and  responding  to  the  target 
radio  checks.  After  the  participant  finished  the  navigation  task,  he/she  was  given  a  10-min  break. 

After  the  break,  the  next  trial  started.  The  participant  was  given  a  different  navigation  map  for 
the  next  environment  and  he/she  was  taken  to  the  starting  point.  The  participant  performed  the 
navigation  task  for  that  environment  and  responded  to  the  target  radio  check  messages.  This 
procedure  was  repeated  for  each  of  the  four  experimental  environments. 

The  participant  performed  a  total  of  four  trials,  one  in  each  of  the  four  environments,  each  with  a 
short  break  afterward.  After  completing  the  navigation  task  in  the  fourth  and  last  environment, 
the  participant  was  brought  to  the  TESF,  where  he/she  completed  the  final  questionnaire 
(appendix  E),  the  experiment  ended,  and  the  participant  was  free  to  leave. 

3.6  Experimental  Design 

For  the  navigation  tasks,  radio  check  tasks,  and  questionnaires,  the  treatment  structure  was  a 
within- subjects  factorial  design  with  one  independent  variable  (environment).  The  four  different 
environments  were  (1)  ARL  Warehouse  Site,  while  wearing  no  rucksack  (Warehouse  Without 
Load),  (2)  ARL  Warehouse  Site,  while  wearing  a  24-kg  (53-lb)  fighting  load  (Warehouse  With 
Load),  (3)  TESF,  while  experiencing  the  virtual  warehouse  environment  shown  on  a  computer 
monitor  (computer  monitor),  and  (4)  TESF  while  experiencing  the  virtual  warehouse 
environment  in  the  CAVE. 
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Experimental  environments  were  assigned  to  participants  by  means  of  a  Williams  latin  square 
design  (Williams,  1949)  to  ensure  that  each  treatment  was  administered  first,  second,  third,  and 
last,  and  no  treatment  consistently  preceded  or  followed  another.  Different  Williams  latin  squares 
were  also  used  to  counterbalance  the  order  of  presentation  of  the  four  navigation  maps  and  the 
four  radio  simulations. 

3.6.1  Dependent  Variables 

The  four  dependent  variables  are  as  follows: 

1.  Navigation  task  data  were  collected  at  each  environment.  The  data  consisted  of: 

a.  Navigation  accuracy  (number  of  participant  response  errors):  these  data  included  (1) 
number  of  times  the  participant  took  a  wrong  turn,  (2)  number  of  times  the  participant 
asked  navigation  directions  from  the  experimenter  or  a  road  guard,  and  (3)  number  of 
times  the  participant  made  a  fatal  navigation  error  (disabled  the  computer  simulation). 

b.  Navigation  time:  the  time  required  by  the  participant  to  travel  from  the  starting  line  to 
the  finish  line  for  each  mapped  navigation  path. 

2.  Radio  check  data  were  collected  in  each  environment.  The  data  consisted  of: 

a.  Radio  check  accuracy  (number  of  participant  response  errors):  these  data  included  (1) 
number  of  times  the  participant  responded  to  an  incorrect  call  sign,  (2)  number  of 
times  the  participant  failed  to  respond  to  a  correct  call  sign,  and  (3)  number  of  times 
the  participant  repeated  or  omitted  words  in  his  or  her  call  sign  response. 

b.  Participant  response  time:  for  correct  radio  check  responses,  the  time  from  the  onset 
of  the  radio  check  request  to  the  onset  of  the  participant’s  response. 

c.  Participant  response  duration:  for  correct  radio  check  responses,  the  time  from  the 
onset  of  the  participant  response  to  the  end  of  the  offset  of  the  final  word  of  the 
participant  response. 

3.  Final  questionnaire  data  for  each  environment.  The  data  consisted  of: 

a.  The  rated  level  of  comfort  and  physical  effort  for  all  environments,  for  both  the  radio 
check  and  navigation  tasks. 

b.  The  ranked  order  of  ease  of  performance  for  all  environments,  for  both  the  radio 
check  and  navigation  tasks. 

4.  Spatial  ability  data  consisted  of: 

a.  Scores  from  the  Perspective  Taking/Spatial  Orientation  Test,  used  as  a  covariate  for 
radio  check  data. 

b.  Scores  from  the  Santa  Barbara  Sense  of  Direction  Scale. 
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3.7  Data  Analysis 

Data  analyses  were  performed  after  the  conclusion  of  the  experiment.  For  the  navigation  times 
and  radio  check  response  times  and  durations,  a  square  root  transformation  was  performed  before 
the  statistical  analysis  to  give  the  data  a  more  normal  distribution  (Bland  and  Altman,  1996).  For 
the  purpose  of  reporting  transformed  data,  and  to  make  results  more  understandable  to  the  reader, 
square  root  means  were  squared  and  then  reported.  Square  root  95%  confidence  limits  were  also 
squared  and  then  reported. 

For  all  dependent  variables  with  interval  data  (navigation  time  and  radio  check  response  time  and 
duration),  data  were  analyzed  separately  using  separate  Linear  Mixed  Model  Analyses  (SPSS 
Version  19).  For  scores  with  sufficient  degrees  of  freedom  (radio  check  response  time  and 
duration),  scores  on  the  perspective  taking/spatial  orientation  test  were  used  as  a  covariate.  For 
all  data,  effects  showing  a  p  value  greater  than  0.05  were  not  considered  statistically  significant. 
A  least  significant  difference  (LSD)  post  hoc  test  was  performed  on  significant  effects,  and 
effects  showing  a  p  value  greater  than  0.05  were  not  considered  statistically  significant. 

For  navigation  accuracy  data,  and  for  questionnaire  ratings  for  comfort,  physical  effort,  and  ease 
of  performance,  the  data  were  analyzed  separately  using  nonparametric  tests.  The  Independent- 
Samples  Kruskal-Wallis  Test  was  used  for  navigation  accuracy  and  for  the  questionnaire  comfort 
and  physical  effort  ratings.  The  Independent-Samples  Chi-Square  Test  was  used  to  analyze 
ranked  ease  of  performance.  For  all  questionnaire  data,  effects  showing  a  p  value  greater  than 
0.05  were  not  considered  statistically  significant. 


4.  Results 


4.1  Participants 

Post-study  statistical  tests  (an  SPSS  Mixed  Linear  Model)  were  performed  on  the  transformed 
navigation  time  and  radio  check  time  and  duration  data,  using  compensation  as  the  only  variable, 
to  determine  whether  there  was  any  significant  difference  with  respect  to  compensation.  Results 
showed  no  significant  navigation  time  differences  between  compensated  and  uncompensated 
participants,  F  (1,  7.541)  =  0.400,  p  =  0.546.  There  were  also  no  significant  differences  between 
compensated  and  uncompensated  participants  for  radio  task  response  time,  F  (1,14)  =  0.667, 
p  =  0.428,  or  for  radio  check  duration,  F  (1,14)  =  2.970,  p  =  0.107.  Thus,  compensated  and 
uncompensated  participants  were  combined  into  one  group  of  participants  for  data  analysis. 

4.2  Navigation  Task  Data 
4.2.1  Navigation  Accuracy 

Navigation  accuracy  data  were  not  available  for  one  participant  (participant  16).  Results  for  the 
remaining  15  participants  indicated  that  there  were  a  total  of  1 1  participant  errors  in  a  total  of  60 
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navigation  tasks  performed  (18%  of  navigation  tasks  had  errors).  Almost  all  navigation  errors 
were  wrong  turns.  An  Independent-Samples  Kruskal  Wallis  Test  indicated  that  there  was  a 
significant  difference  (df  =  3,  61;  H  =  9.901,  p  =  0.019)  between  the  distribution  of  the  number 
of  errors  in  different  environments;  six  errors  occurred  in  the  Computer  Monitor  environment, 
four  errors  in  the  CAVE,  one  error  in  the  Warehouse  With  Load  environment,  and  none  in  the 
Warehouse  Without  Load  environment.  Thus,  there  were  significantly  more  navigation  errors 
associated  with  virtual  environments  (10  errors  in  the  Computer  and  the  CAVE  environments) 
than  with  the  real  (both  warehouse)  environments  (one  error).  There  were  no  instances  in  which 
the  participant  asked  for  navigation  directions  from  the  experimenter  or  traffic  assistant;  thus,  it 
appears  that  participants  preferred  to  make  navigation  errors  rather  than  ask  for  directions. 

4.2.2  Navigation  Time 

The  SPSS  Linear  Mixed  Model  performed  on  the  transformed  navigation  time  data  from 
navigation  trials  with  no  errors  indicated  a  significant  main  effect  for  environment, 

F  (3,  25.696)  =  15.859,  p  <  0.001.  Lor  reporting  purposes,  the  mean  navigation  times  were 
transformed  back  into  real  (non-transformed)  units,  with  means,  and  upper  and  lower  95% 
confidence  limits  shown  in  table  1.  Ligure  9  shows  mean  navigation  time  as  a  function  of 
environment.  Post  hoc  LSD  tests  indicated  that  mean  navigation  time  for  the  CAVE  was 
significantly  longer  than  all  other  environments  and  the  navigation  time  for  Warehouse  Without 
Load  was  significantly  shorter  than  all  other  environments.  The  Computer  Monitor  and 
Warehouse  With  Load  environments  were  not  significantly  different  from  each  other  (p  =  0.576); 
therefore,  there  was  no  significant  difference  between  virtual  and  real  environments.  The  largest 
difference  between  means  (between  the  CAVE  and  the  Warehouse  Without  Load  environments) 
was  1.228  min. 

Table  1.  Mean  navigation  times3  and  95%  confidence  intervals,  as  a  function  of  environment. 


Environment 

Mean  Navigation 
Time 
(min) 

95%  Confidence  Interval 

95%  Confidence  Interval 

Lower  Bound 

Upper  Bound 

Warehouse  Without 

Load 

10.407  a 

10.068 

10.752 

Warehouse  With  Load 

10.930  b 

10.582 

11.290 

Computer  Monitor 

11.036  b 

10.654 

11.424 

CAVE 

11.635  c 

11.256 

12.020 

Means  with  different  letters  are  significantly  different. 
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Environment 
Error  Bars:  95%  Cl 

Figure  9.  Mean  navigation  time  (in  minutes)  with  95% 
confidence  intervals,  as  a  function  of 
environment. 

4.3  Radio  Check  Data 
4.3.1  Radio  Check  Accuracy 

Results  indicated  that  participant  response  to  target  radio  check  messages  was  very  accurate, 
showing  a  total  of  25  errors  (4%)  out  of  a  total  of  640  total  radio  check  tasks.  Figure  10  shows 
the  types  and  number  of  errors  for  each  environment.  Nine  errors  consisted  of  no  participant 
response  to  the  target  call  sign,  six  errors  were  use  of  an  improper  word  other  than  call  sign  in 
the  radio  check  response,  and  four  errors  concerned  the  use  of  an  improper  call  sign  in  the  radio 
check  response.  In  examining  number  of  errors  as  a  function  of  environment,  a  total  of  14  errors 
were  made  in  real  environments:  8  in  the  Warehouse  Without  Load  and  6  in  the  Warehouse  With 
Load.  The  remaining  1 1  errors  were  made  in  virtual  environments:  9  in  the  CAVE  and  2  at  the 
Computer  Monitor.  However,  a  Chi-Square  Test  indicated  that  there  was  no  significant 
difference  for  number  of  errors  in  each  environment  (df  =  3,  x  =  1.372,  p  =  0.712). 
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Figure  10.  Type  and  number  of  errors  as  a  function  of 
environment. 

4.3.2  Radio  Check  Response  Time 

The  SPSS  Linear  Mixed  Model  performed  on  the  transformed  data,  with  scores  on  the 
perspective  taking/spatial  orientation  test  used  as  a  covariate,  indicated  a  significant  main  effect 
for  environment,  F(3,  563.302)  =  13.097,  p  =  0.001.  For  reporting  purposes,  the  mean  radio 
check  response  times  transformed  back  into  nontransformed  units  with  means,  and  upper  and 
lower  confidence  limits,  are  shown  in  table  2.  Post  hoc  LSD  tests  (p  <  0.001)  indicated  that  the 
mean  radio  check  response  time  for  the  Warehouse  With  Load  environment  was  significantly 
shorter  than  all  other  environments.  There  were  no  other  significant  differences.  Thus,  there  was 
no  significant  difference  in  mean  response  time  between  the  CAVE,  Computer  Monitor,  or 
Warehouse  Without  Load  environments.  The  largest  significant  difference  between  means 
(between  the  CAVE  and  the  Warehouse  With  Load  and  environments)  was  0.109  sec. 

Table  2.  Mean  radio  check  response  times3  and  95%  confidence  intervals,  as  a  function  of  environment. 


Environment 

Mean  Radio 
Check  Response 
Time 

(s) 

95%  Confidence  Interval 
Lower  Bound 

95%  Confidence  Interval 
Upper  Bound 

Warehouse  With  Load 

5.126  a 

5.049 

5.203 

Warehouse  Without 

Load 

5.198b 

5.121 

5.276 

Computer  Monitor 

5.189b 

5.117 

5.267 

CAVE 

5.235  b 

5.162 

5.313 

“Means  with  different  letters  are  significantly  different. 
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4.3.3  Radio  Check  Response  Duration 

The  SPSS  Linear  Mixed  Model  performed  on  the  transformed  data,  with  perspective 
taking/spatial  orientation  test  scores  used  as  a  covariate,  indicated  a  significant  main  effect  for 
environment,  F  (3,  569.898)  =  5.076,  p  =  0.002.  The  mean  radio  check  task  response  durations 
with  means  and  upper  and  lower  confidence  limits  are  shown  in  table  3.  Post  hoc  LSD  tests  (p  < 
0.05)  indicated  that  the  CAVE  response  duration  was  significantly  greater  than  all  other 
environments.  Thus,  there  was  no  significant  difference  between  the  virtual  Computer  Monitor 
environment  and  the  real  (Warehouse  With  Load  and  Warehouse  without  Load)  environments. 
The  largest  significant  difference  between  means  (between  the  Computer  Monitor  and  the  CAVE 
environments)  was  0.063  sec. 


Table  3.  Mean  radio  check  response  durations3  and  95%  confidence  intervals,  as  a  function  of  environment. 


Environment 

Mean  Radio 
Check  Response 
Duration 

(s) 

95%  Confidence  Interval 
Lower  Bound 

95%  Confidence  Interval 
Upper  Bound 

Computer  Monitor 

2.729  a 

2.544 

2.921 

Warehouse  With  Load 

2.732  a 

2.547 

2.924 

Warehouse  Without 

Load 

2.749  a 

2.563 

2.941 

CAVE 

2.792  b 

2.605 

2.986 

“Means  with  different  letters  are  significantly  different. 


4.4  Final  Questionnaire  Data 

4.4.1  Comfort 

An  Independent- Samples  Kruskal-Wallis  test  for  median  comfort  rating  as  a  function  of  radio 
check  and  navigation  tasks  indicated  no  significant  differences  for  environment  ( df=  3,  127, 

H  =  4.509,  p  >  0.05),  but  did  report  a  significant  effect  for  task  type  (radio  check  and  navigation 
tasks)  {df  =  1,  127,  H  =  4.874,  p  =  0.027).  The  data  showed  that  the  median  rank  for  comfort 
associated  with  the  radio  check  task  was  70.83,  while  the  median  rank  for  comfort  associated 
with  the  navigation  task  was  57.28,  indicating  that  the  navigation  task  had  a  significantly  greater 
number  of  lower  comfort  ratings  than  did  the  audio  task. 

4.4.2  Physical  Effort 

The  Independent-Samples  Kruskal-Wallis  Test  performed  on  the  level  of  physical  effort  ratings 
for  navigation  tasks  indicated  that  environment  was  significant  {df  =  3,  127,  H  =  44.121, 
p  <  0.001).  Larger  means  denote  higher  levels  of  perceived  physical  effort.  The  Kruskal-Wallis 
data  indicated  that  mean  ranks  were  45.69  for  CAVE,  50.42  for  Computer  Monitor,  63.36  for 
Warehouse  Without  Load,  and  95.95  for  Warehouse  With  Load.  Figure  1 1  shows  box  plots 
containing  the  medians  and  95%  confidence  limits  for  all  environments.  The  rankings  indicated 
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that  Warehouse  With  Load  was  rated  as  requiring  a  significantly  higher  level  of  physical  effort 
(p  >  0.05)  than  any  other  environment.  There  was  no  significant  difference  between  the  virtual 
environments. 


Independent-Samples  Kruskal-Wallis  Test 


Figure  1 1 .  Physical  effort  questionnaire  rating  as 
a  function  of  environment  with  95% 
confidence  intervals. 

4.4.3  Ranked  Ease  of  Performance  for  the  Navigation  Task 

To  obtain  navigation  task  data,  participants  were  asked  to  rank  each  environment  for  ease  of 
performing  the  navigation  task;  a  rank  of  1  was  easiest  for  navigating,  2  was  more  difficult, 

3  was  more  difficult  than  2,  and  4  was  the  most  difficult.  A  Chi-Square  Test  performed  on  the 
ranking  data  indicated  that  there  were  significant  differences  in  environment  rankings  (df  =  9, 
Pearson  X  =  34.143,  p  <  0.005).  As  shown  in  table  4,  the  Warehouse  Without  Load  environment 
had  the  highest  number  (1 1)  of  1  rankings,  indicating  the  greatest  ease  of  performance.  The 
Computer  Monitor  environment  had  the  greatest  number  (7)  of  2  rankings.  Warehouse  Without 
Load  and  CAVE  each  had  the  greatest  number  (5)  of  3  rankings.  The  Warehouse  With  Load 
environment  was  ranked  fourth  (indicating  lowest  ease  of  performance)  nine  times.  The  data 
indicated  that  the  Warehouse  Without  Load  was  ranked  highest  for  ease  of  performance  while 
Warehouse  with  Load  was  ranked  lowest. 
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Table  4.  Count  and  expected  count  for  ranked  ease  of  navigation  task  performance 
in  each  environment. 


Environment 

Ranking 

Total 

1 

2 

3 

4 

Virtual  in 

CAVE 

Count 

7 

3 

5 

2 

17 

Expected  Count 

4.75 

4.00 

4.25 

4 

17 

Computer 

Monitor 

Count 

1 

7 

4 

5 

17 

Expected  Count 

4.75 

4.00 

4.25 

4 

17 

W  arehouse 

Without 

Load 

Count 

11 

1 

5 

0 

17 

Expected  Count 

4.75 

4.00 

4.25 

4 

17 

W  arehouse 

With  Load 

Count 

0 

5 

3 

9 

17 

Expected  Count 

4.75 

4.00 

4.25 

4 

17 

Total 

Count 

19 

16 

17 

16 

68 

Expected  Count 

19 

16 

17. 

16 

68 

4.4.4  Ranked  Ease  of  Performance  for  the  Radio  Check  Task 

To  obtain  the  radio  check  task  data,  participants  were  asked  to  rank  each  environment  for  ease  of 
performing  the  task;  a  rank  of  1  was  easiest,  2  was  more  difficult,  3  was  more  difficult  than  2, 
and  4  was  the  most  difficult.  A  Chi-Square  Test  performed  on  ranking  data  indicated  that  there 
were  significant  differences  in  environment  rankings  (df  =  9,  Pearson  X  =  37.328,  p  <  0.005).  As 
shown  in  table  5,  the  Warehouse  Without  Load  environment  had  the  highest  number  (10)  of  1 
rankings,  indicating  the  greatest  ease  of  performance;  the  Computer  Monitor  environment  had 
the  greatest  number  (9)  of  2  rankings;  Warehouse  Without  Load  had  the  greatest  number  (6)  of  3 
rankings;  and  the  Warehouse  With  Load  environment  was  ranked  fourth  (indicating  lowest  ease 
of  performance)  10  times.  The  data  indicated  that  the  Warehouse  without  Load  was  ranked 
highest  for  ease  of  performance  and  Warehouse  With  Load  was  ranked  lowest. 
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Table  5.  Count  and  expected  count  for  ranked  ease  of  radio  check  task  performance 
in  each  environment 


If  riiTivrtniYiQiil 

Ranking 

Total 

1 

2 

3 

4 

Virtual  in 

Count 

7 

2 

4 

3 

16 

CAVE 

Expected  Count 

4.538 

3.821 

3.821 

3.820 

16 

Computer 

Count 

1 

9 

4 

3 

17 

Monitor 

Expected  Count 

4.820 

4.060 

4.060 

4.060 

17 

Warehouse 

Count 

10 

1 

6 

0 

17 

Without  Load 

Expected  Count 

4.820 

4.060 

4.060 

4.060 

17 

Warehouse 

Count 

1 

4 

2 

10 

17 

With  Load 

Expected  Count 

4.820 

4.060 

4.060 

4.060 

17 

Total 

Count 

19 

16 

16 

16 

68 

Expected  Count 

19 

16 

16 

16 

127.0 

5.  Discussion 


The  objective  of  this  study  was  to  develop  a  characterization  of  the  interaction  of  physical  load 
and  environment  on  dismounted  Soldier  performance  in  navigation  and  radio  check  tasks.  This 
characterization  was  accomplished  by  exploring  the  effect  of  physical  load  on  Soldier 
performance,  investigating  the  effect  of  real  and  virtual  environments  on  Soldier  performance, 
and  exploring  the  potential  of  radio  check  response  time,  duration,  and  accuracy  as  measures  of 
dismounted  Soldier  cognitive  performance. 

5.1  The  Effect  of  Physical  Load  on  Soldier  Performance 

It  was  hypothesized  that  Soldier  performance  on  navigation  and  radio  check  tasks  would  be 
negatively  correlated  with  load,  and  that  Soldier  performance  would  show  some  degradation 
from  the  no-load  to  the  24-kg  load  environment.  Thus,  the  experimenters  expected  that 
performance  time  means  for  the  24-kg  load  environment  would  be  significantly  greater  than 
means  for  the  no-load  environments. 
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The  results  of  this  study  support  the  hypothesis.  Soldier  load  did  have  a  significant  effect  on 
navigation  time;  navigation  time  was  shorter  for  navigating  the  warehouse  with  no  load  than  for 
any  other  condition.  Subjective  questionnaire  data  indicated  that  navigating  with  a  load  resulted 
in  higher  perceived  physical  effort  and  lower  ease  of  performance.  Overall,  the  results  indicated 
that  physical  load  did  have  a  significant,  consistent  effect  on  Soldier  performance. 

For  radio  check  response  time,  the  Warehouse  With  Load  was  significantly  faster  than  any  other 
environment.  The  lack  of  performance  decrement  with  increased  load  was  explained  by  Crowell 
et  al.  (1999),  who  suggested  that  carrying  a  light  load  (in  that  case,  23  kg)  might  cause  an 
exercise-induced  increase  in  alertness,  which  could  result  in  improved  performance  of  a 
monitoring  task.  The  data  from  this  study  also  agree  with  the  results  of  studies  reviewed  by 
Tomporowski  (2003),  who  found  that  acute  bouts  of  exercise  can  facilitate  response  speed  and 
accuracy. 

5.2  The  Effect  of  Real  and  Virtual  Environments  on  Soldier  Cognitive  and  Physical 
Performance 

It  was  hypothesized  that  navigation  in  both  virtual  environments  (CAVE  and  Computer  Monitor) 
would  require  more  time  and  attention  than  navigation  in  both  real  environments  (Warehouse 
With  Load  and  the  Warehouse  Without  Load).  Thus,  it  was  expected  that  means  for  the  virtual 
environments  would  be  significantly  greater  than  means  for  the  real-world  environments. 

Overall,  the  quantitative  data  showed  that  real  and  virtual  environments  had  a  significant  effect 
on  navigation  time.  It  took  Soldiers  longer  to  navigate  in  both  of  the  virtual  environments  than  to 
walk  around  the  warehouses  without  load.  This  is  also  reflected  in  the  rankings  for  ease  of 
performance  of  the  navigation  task.  Soldiers  ranked  the  virtual  environments  lower  than  the 
Warehouse  Without  Load  condition.  In  addition,  the  navigation  time  in  the  CAVE  was 
significantly  longer  than  at  the  computer  monitor.  Similarly,  environment  had  a  significant  effect 
on  navigation  accuracy.  Significantly  more  errors  occurred  in  the  virtual  environment  than  the 
real  environment.  This  is  similar  to  the  result  found  by  Schmelter  et  al.  (2009)  for  subjects 
trained  to  navigate  a  maze.  Subjects  who  navigated  the  virtual  maze  made  more  mistakes  than 
those  who  navigated  the  real  maze.  In  contrast  to  the  results  for  the  navigation  task,  environment 
affects  only  part  of  the  radio  check  task.  Environment  affects  response  duration  but  not  response 
accuracy  or  response  time.  The  response  duration  is  longer  in  the  CAVE  than  any  of  the 
environments. 

The  lagging  CAVE  performance  in  the  navigation  time  and  radio  check  duration  tasks  may 
partially  be  explained  by  Klein  (1976),  who  suggested  that  “moving  per  se  does  not  always 
require  attention,  rather  it  is  the  guidance  of  movement,  in  terms  of  direction  and  speed  that 
requires  attention.”  It  could  be  possible  that  the  guidance  of  movement  (using  a  joystick)  was 
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more  cognitively  demanding  in  the  CAVE,  and  that  the  extent  to  which  movement  itself,  or  the 
need  to  attend  to  virtual  movement,  imposed  a  cognitive  demand  that  was  greater  than  that 
required  by  the  warehouse  environments.  However,  Klein’s  suggestion  does  not  explain  why 
CAVE  navigation  time  and  radio  check  duration  were  significantly  longer  than  those  for  the 
Computer  Monitor  environment,  in  which  Soldiers  also  used  a  joystick  to  guide  movement. 
Future  research  should  be  conducted  to  explain  this  discrepancy. 

5.3  The  Potential  of  Radio  Check  Response  Time,  Duration,  and  Accuracy  as  Measures  of 
Dismounted  Soldier  Cognitive  Performance 

Because  of  the  relationship  between  cognitive  performance  and  speech  measures,  it  was 
hypothesized  that  radio  check  response  time,  duration,  and  accuracy  would  have  the  potential  for 
reflecting  meaningful  differences  in  Soldier  performance. 

Response  accuracy  can  be  used  as  a  measure  of  cognitive  performance  because  it  is  not  affected 
by  the  load  carried  or  the  environment.  Response  time  and  response  duration  can  also  be  used  as 
measures  of  cognitive  performance  provided  two  limitations  are  taken  into  consideration:  (1) 
response  time  is  shorter  when  subjects  carry  a  load  and  (2)  response  duration  in  the  CAVE  is 
longer  than  in  the  other  environments.  Therefore,  accuracy,  response  time,  and  response  duration 
can  be  used  as  metrics  of  cognitive  performance  provided  that  (1)  response  time  results  are  not 
compared  if  the  load  carried  was  different  and  (2)  results  for  response  duration  are  not  compared 
between  the  CAVE  environment  and  any  other  environment. 

The  results  of  this  study  showed  several  significant  differences  for  radio  check  response  time  and 
response  duration  means,  although  differences  between  the  highest  and  lowest  means  were  often 
tenths  of  seconds.  However,  response  times  and  durations  of  this  nature  may  have  practical  as 
well  as  statistical  significance.  In  modern  missions,  dismounted  Soldiers  are  required  to  perform 
tasks  in  threatening  or  ambiguous  environments  (e.g.,  patrolling  potentially  hostile  city  streets  to 
detect  dismounted  enemy  snipers).  In  these  environments,  decisions  to  react  (for  example, 
whether  an  object  just  spotted  could  be  a  friendly  ally  or  an  unfriendly  enemy  target)  must  be 
made  in  very  short  periods  of  time.  Because  Soldier  decision  time  may  need  to  be  very  short,  the 
mean  radio  check  response  times  and  durations  may  have  practical  significance  for  the 
dismounted  Soldier. 


6.  Conclusions  and  Recommendations 


In  short,  results  from  this  research  indicated  that  physical  load  and  environment  each  affect 
navigation  and  responses  to  a  radio  check  task.  For  the  conditions  in  this  study,  carrying  a  load 
increased  navigation  time  but  decreased  response  time  for  the  radio  check  task.  With  regard  to 
the  environment  in  which  an  experiment  is  conducted,  more  navigation  mistakes  occurred  in  the 
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virtual  environment  than  in  the  real  environment.  In  addition,  navigating  in  the  CAVE  takes 
longer  than  in  the  other  environments  (real  world  and  Computer  Monitor).  Responses  to  the 
radio  check  task  are  also  longer  in  the  CAVE  than  in  the  other  environments. 

Three  different  recommendations  can  be  made  based  on  the  results  of  this  study: 

•  Explore  the  effect  of  heavier  loads  on  soldier  performance.  Results  from  this  study 
indicated  that  physical  exertion,  specifically  the  weight  of  the  rucksack,  had  a  discernible 
effect  on  Soldier  physical  and  cognitive  performance.  The  load  used  in  this  study  was  a 
relatively  light  fighting  load  of  24  kg  packed  inside  a  U.S.  Army  rucksack.  In  addition,  the 
navigation  task  was  relatively  simple,  involving  a  short  (approximately  10-min)  walking- 
paced  task  performed  upon  predominately  level  concrete  roads.  However,  the  24-kg  load 
does  not  represent  the  load  that  the  dismounted  Soldier  must  often  carry  in  battlefield 
environments.  Bleidel  (2011)  observed  that  Soldiers  often  perform  tasks  while  carrying 
heavy  loads  (about  130  lb)  for  several  hours.  Researchers  using  heavy  loads  such  as  these 
(May  et  al.,  2009)  found  that  carriage  load  disrupted  balance  control  and  degraded  both 
cognitive  processes  and  situational  awareness.  Future  research  should  involve  more 
realistic  criteria,  including  greater  carriage  loads  and  longer  task  times,  to  determine  the 
effect  of  physical  load  on  Soldier  performance. 

•  Use  additional  radio  check  task  metrics  to  measure  Soldier  cognitive  performance.  The 
radio  check  task  used  in  this  study  was  developed  to  measure  the  effects  of  cognitive  load 
on  Soldier  speech.  Response  time,  duration,  and  accuracy  are  measures  of  speech  fluency 
and  speech  rate,  which  are  affected  by  auditory  cognitive  and  attentional  demands  (Scherer 
et  al.,  2002).  The  results  from  this  study  indicated  that  radio  check  response  accuracy  can 
be  a  useful  metric  of  Soldier  cognitive  performance.  Response  time  and  duration  can  also 
be  used  to  assess  cognitive  performance  if  comparisons  between  conditions  are  made  only 
when  load  is  constant  and  when  the  results  for  the  CAVE  environment  are  not  compared  to 
the  results  from  other  environments.  However,  additional  radio  check  metrics  have  been 
suggested;  Scherer  et  al.  (2002)  noted  that  fundamental  frequency,  speech  energy  contours, 
and  spectral  parameters  can  also  be  used  as  speech  metrics.  Therefore,  future  studies 
should  include  these  metrics. 

•  Explain  performance  differences  found  between  the  CAVE  and  other  environments. 

Soldier  performance  in  the  radio  check  and  navigation  tasks  indicated  that  performance  in 
the  CAVE  lagged  behind  that  of  the  real-world  warehouse  environments  and  sometimes 
behind  the  Computer  Monitor  environment.  This  may  have  been  due  to  some  factor  in  the 
simulation.  By  definition,  a  simulation  in  some  way  always  deviates  from  the  real  world. 
However,  the  deviation  from  reality  means  that  some  aspects  of  the  simulation  may  not 
transfer  to  the  real  world,  and  these  dissimilarities  might  affect  some  aspect  of  human 
performance.  Philip  et  al.  (2005)  found  that  reaction  times  were  slower  during  a  simulated 
driving  task  than  in  the  real  world.  Bishop  and  Rohrmann  (2003)  also  found  that  even 
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detailed  computer  simulations  do  not  necessarily  generate  the  same  quantitative  or 
subjective  responses  as  the  corresponding  real-world  urban  environment.  It  is 
recommended  that  future  research  be  conducted  to  determine  precisely  why  the  virtual 
CAVE  had  a  more  detrimental  effect  on  Soldier  performance  than  did  the  Warehouse  With 
Load  environment. 
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Appendix  A.  Participant  Informed  Consent  Form 


This  appendix  appears  in  its  original  form,  without  editorial  change. 


Consent  Form 

Army  Research  Laboratory,  Human  Research  &  Engineering  Directorate 
Aberdeen  Proving  Ground,  MD  21005 


Title  of  Project:  Measures  of  the  Ability  of  Simulated  Environments  to  Replicate  Effects  of 
Physical  and  Cognitive  Stress 

Project  Number:  ARL- 11-041 


Sponsor:  Army  Research  Laboratory 


Principal  Investigator:  Ellen  C.  Haas 

Army  Research  Laboratory 

Human  Research  &  Engineering  Directorate 

RDRL-HRS-B,  Bldg.  5 17A 

Aberdeen  Proving  Ground,  MD  21005 

(410)  278-5825;  ellen.c.haas.civ@us.army.mil 


Associate  Investigator:  H.  Philip  Crowell 

Army  Research  Laboratory 

Human  Research  &  Engineering  Directorate 

RDRL-HRS-B,  Bldg.  5 17A 

Aberdeen  Proving  Ground,  MD  21005 

(410)  278-5986;  harrison.philip.crowell@us.army.mil 

Kathy  L.  Kehring 

Army  Research  Laboratory 

Human  Research  &  Engineering  Directorate 

RDRL-HRS-B,  Bldg.  518 

Aberdeen  Proving  Ground,  MD  21005 

(410)  278-5894;  kathy.kehring@us.army.mil 


You  are  being  asked  to  join  a  research  study.  This  consent  form  explains  the  research  study  and 
your  part  in  it.  Please  read  this  form  carefully  before  you  decide  to  take  part.  You  can  take  as 
much  time  as  you  need.  Please  ask  the  research  staff  any  questions  at  any  time  about  anything 
you  do  not  understand.  You  are  a  volunteer.  If  you  join  the  study,  you  can  change  your  mind 
later.  You  can  decide  not  to  take  part  now  or  you  can  quit  at  any  time  later  on. 
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Location  of  the  Study 

Aberdeen  Proving  Ground,  MD,  Building  518. 

Purpose  of  the  Study 

The  purpose  of  this  study  is  to  determine  whether  cognitive  tasks  and  physiological  load  affects 
user  performance  in  a  map  navigation  task. 

Procedures  to  be  Followed 

You  will  be  exposed  to  four  different  experimental  conditions  in  which  you  will  perform  a 
navigation  task  using  a  map.  At  the  same  time,  you  will  listen  for  auditory  call  signs  and  respond 
when  your  particular  call  sign  is  given.  At  the  end  of  the  experiment,  you  will  be  given  a  final 
questionnaire  in  which  you  can  give  comments  on  the  call  sign  task,  and  the  ease  of  use  of 
performing  each  task.  You  can  have  a  break  between  each  task. 

Tasks/Training  Before  the  Study  Begins: 

First,  we  will  ask  you  about  your  medical  status  (any  injuries  or  illnesses).  Next,  we  will  perform 
a  Snellen  eye  test  in  which  we  will  check  your  vision  to  make  sure  that  it  is  normal  (20:30  vision, 
which  can  include  wearing  corrective  lenses,  such  as  i.e.,  eyeglasses  or  contact  lenses).  You  may 
not  participate  in  this  study  if  you  have  an  illness  or  injury  that  would  affect  your  ability  to  see  or 
hear,  or  have  vision  that  does  not  fulfill  the  20:30  vision  criteria. 

Next,  you  will  fill  out  a  demographic  questionnaire  that  asks  about  your  gender,  age,  and  military 
service  (if  applicable),  as  well  about  other  personal  characteristics.  Next,  you  will  fill  out  a 
computer  questionnaire  that  asks  about  your  computer  experience  (number  of  hours  spent  on  a 
computer,  computer  games  that  you  play).  After  that,  you  will  fill  out  two  questionnaires  that 
ask  about  your  spatial  ability. 


After  this,  you  will  perform  experimental  trials.  Prior  to  each  experimental  condition,  we  will 
show  you  how  to  perform  the  call  sign  and  the  navigation  task,  and,  if  applicable,  the  computer 
task  used  in  that  condition. 


Experimental  Trials:  After  pre-experimental  training  is  over  for  that  condition,  you  will 
perform  four  experimental  conditions;  1)  Using  a  map  to  navigate  (walking)  around  ARL 
warehouses  while  wearing  no  physical  load;  2)  Using  a  map  to  navigate(walking)  around  ARL 
warehouses  while  wearing  a  fighting  load  53  lbs  (simulated  by  using  weights  contained  in  a 
backpack  worn  on  the  your  back;  3)  In  Bldg.  518,  using  a  map  and  joystick  to  navigate  (sitting) 
around  a  computerized  version  of  the  warehouse,  as  shown  on  a  computer  monitor  in  Bldg.  518, 
and  4)  Using  a  map  and  joystick  to  navigate  (sitting)  through  a  virtual  warehouse  environment 


33 


viewed  on  four  screens  of  the  Immersive  Environment  Simulator  (IES).  During  the  tasks  in 
Bldg.  518,  your  travel  speed  will  be  capped  at  4.0  mph.  During  each  of  these  tasks,  you  will 
listen  to  military  call  signs  played  on  audio  MP3  players,  and  will  verbally  respond  when  you 
hear  your  call  sign. 

After  the  trials  are  over,  you  will  be  asked  to  fill  in  another  questionnaire  that  deals  with  how 
you  felt  about  navigating  in  each  environment. 

Duration  (Time  Required) 

The  study  will  require  approximately  4  hours  of  your  time  over  a  1  day  period.  Participants  from 
A-Team  Solutions  will  require  an  additional  30  minutes  for  pick-up  and  drop-off  at  the  APG 
Burger  King  meeting  site. 

Compensation  for  Participation 

You  cannot  be  paid  if  you  are  a  member  of  the  military,  a  civilian  employee  of  the  U.  S. 
Government,  or  a  family  member  of  an  employee  of  the  Human  Research  &  Engineering 
Directorate. 

Non-government  civilian  participants  will  be  reimbursed  for  their  time  at  a  rate  of  $30  per  hour. 
Participants  dismissed  early  (due  to  voluntary  withdrawal  or  dismissal)  will  be  paid 
commensurate  with  the  time  completed  at  a  minimum  of  one  half-hour  compensation.  If  you  are 
working  for  a  temporary  agency,  you  will  be  paid  directly  by  that  agency.  Payment  forms  are 
completed  for  each  monetary  disbursement  that  provide  data  about  the  volunteer  to  the  Defense 
Finance  and  Accounting  Office  (DFAS). 

Risks 

The  investigators  will  monitor  the  safety  of  the  volunteers  in  the  study;  however,  not  all  risks  or 
discomforts  can  be  eliminated.  Risks  and  discomforts  include  those  commonly  associated  with 
using  computer  joystick  controls,  such  as  fatigue. 

Benefits 

Your  vision  will  be  checked  for  free,  and  you  may  feel  satisfaction  for  participating  in  a  study  to 
identify  the  most  appropriate  environment  for  specific  types  of  dismounted  Soldier  research. 

Confidentiality 

Your  participation  in  this  research  is  confidential.  The  data  will  be  stored  and  secured  at 
Aberdeen  Proving  Ground,  in  a  locked  file  cabinet.  The  data,  without  any  identifying 
information,  will  be  transferred  to  a  password-protected  computer  for  data  analysis.  After  the 
data  is  put  in  the  computer  file,  any  paper  copies  of  the  data  will  be  shredded.  This  consent  form 
will  be  retained  by  the  principal  investigator  for  three  years. 
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If  the  results  of  the  experiment  are  published  or  presented  to  anyone,  no  personally  identifiable 
information  will  be  shared.  Publication  of  the  results  of  this  study  in  a  journal  or  technical  report, 
or  presentation  at  a  meeting,  will  not  reveal  personally  identifiable  information.  The  research 
staff  will  protect  your  data  from  disclosure  to  people  not  connected  with  the  study.  However, 
complete  confidentiality  cannot  be  guaranteed  because  officials  of  the  U.  S.  Army  Human 
Research  Protections  Office  and  the  Army  Research  Laboratory’s  Institutional  Review  Board  are 
permitted  by  law  to  inspect  the  records  obtained  in  this  study  to  insure  compliance  with  laws  and 
regulations  covering  experiments  using  human  subjects. 

We  would  like  your  permission  to  record  your  voice  during  the  experimental  session.  Recordings 
of  your  voice  will  be  used  in  data  analysis  of  response  time  of  your  speech  commands,  and  will 
not  be  played  at  any  public  event.  If  we  cannot  record  your  voice,  you  cannot  take  part  in  this 
study.  Please  indicate  below  if  you  will  agree  to  allow  us  to  take  pictures  of  you  and  record  your 
voice. 

I  give  consent  have  my  voice  recorded  during  this  study:  _ _Yes  _ No  Please  initial: _ 

Contact  Information  for  Additional  Questions 

You  have  the  right  to  obtain  answers  to  any  questions  you  might  have  about  this  research  both 
while  you  take  part  in  the  study  and  after  you  leave  the  research  site.  Please  contact  anyone  listed 
at  the  top  of  the  first  page  of  this  consent  form  for  more  information  about  this  study.  You  may 
also  contact  the  Chairperson  of  the  Human  Research  &  Engineering  Directorate,  Institution 
Review  Board,  at  (410)  278-5992  with  questions,  complaints,  or  concerns  about  this  research,  or 
if  you  feel  this  study  has  harmed  you.  The  Chairperson  can  also  answer  questions  about  your 
rights  as  a  research  participant.  You  may  also  call  the  Chairperson’s  number  if  you  cannot  reach 
the  research  team  or  wish  to  talk  to  someone  else. 

Voluntary  Participation 

Your  decision  to  be  in  this  research  is  voluntary.  You  can  stop  at  any  time.  You  do  not  have  to 
answer  any  questions  you  do  not  want  to  answer.  Refusal  to  take  part  in  or  withdrawal  from  this 
study  will  involve  no  penalty  or  loss  of  benefits  you  would  receive  by  staying  in  it. 

Military  personnel  cannot  be  punished  under  the  Uniform  Code  of  Military  Justice  for  choosing 
not  to  take  part  in  or  withdrawing  from  this  study,  and  cannot  receive  administrative  sanctions 
for  choosing  not  to  participate.  Civilian  or  contractor  personnel  cannot  receive  administrative 
sanctions  for  choosing  not  to  participate  in  or  withdrawing  from  this  study. 

You  must  be  18  years  of  age  or  older  to  take  part  in  this  research  study.  If  you  agree  to  take  part 
in  this  research  study  based  on  the  information  outlined  above,  please  sign  your  name  and 
indicate  the  date  below. 

You  will  be  given  a  copy  of  this  consent  form  for  your  records. 
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This  consent  form  will  be  stored  by  the  Principal  Investigator  in  a  locked  file  cabinet  for  a 
minimum  of  three  years.  Then  it  will  be  shredded. 


This  consent  form  is  approved  from  7/9/2012  to  7/9/2013. 


Do  not  sign  after  the  expiration  date  of  7/9/2013. 


Participant’s  Signature 


Date 


Participant’s  Printed  Name 


Signature  of  Person  Obtaining  Consent 


Date 


Printed  Name  of  Person  Obtaining  Consent 
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Appendix  B.  Participant  Data  Form 


This  appendix  appears  in  its  original  form,  without  editorial  change. 
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Questionnaire  1:  Subject  Data  Form 

Participant  ID _  Age: . _ 

Gender _ 

If  Military:  MOS: _  Rank _ 

Years  of  Experience: _ 

Do  you  have  prior  military  service?  Yes  No 

If  Yes,  how  long _ 

Health  Screening  Data: 

1.  Do  you  have  any  hearing  problems  such  as  ringing  or  buzzing  in  your  ears?  Yes 

2.  Do  you  currently  have  an  ear  infection?  Yes  No 

3.  Do  you  have  any  vision  problems?  Yes  No 

If  so,  what  are  they?  _ 

4.  Do  you  wear  glasses  when  working  on  the  computer?  Yes  No 

If  YES,  did  you  bring  your  glasses  with  you  today?  Yes  No 

5.  How  many  hours  of  sleep  did  you  get  last  night?  _ _ hours 

6.  Are  you  in  your  usual  state  of  health  physically?  Yes  No 
If  NO,  please  briefly  explain: 


No 
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Questionnaire  2: 

Computer  Use  Questions: 

1 .  How  many  hours  per  week  do  you  use  a  computer  at  home  and  at  work? 

2.  How  many  years  have  you  used  a  computer?  _ 


3.  Which  of  the  following  best  describes  your  expertise  with  computer?  (check  V  one) 
_ _  Novice 

_ Good  with  one  type  of  software  package  (such  as  word  processing  or  slides) 

_ Good  with  several  software  packages 

_ _  Can  program  in  one  language  and  use  several  software  packages 

_ Can  program  in  several  languages  and  use  several  software  packages 


4.  How  often  do  you  play  video  games?  (Circle  one) 

Never  Rarely  Once  Every  Few  Months  Monthly  Daily 

5.  If  you  do  play  video  games  at  least  once  every  few  months,  which  games  do  you  most  often 
play? 

6.  Have  you  ever  used  a  joystick  when  playing  video  games?  Yes  No 

7.  If  you  have  used  a  joystick,  how  many  years  have  you  used  one?  _ _ 

8.  Do  you  consider  yourself  to  be  good  at  working  with  your  hands?  For  example,  good  at 
woodworking  or  sewing?  Yes  No 

9.  Do  you  consider  yourself  to  have  good  hand-eye  coordination?  Yes  No 
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Intentionally  left  blank. 
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Appendix  C.  Santa  Barbara  Sense-of-Direction  Scale 


This  appendix  appears  in  its  original  form,  without  editorial  change. 
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This  questionnaire  consists  of  several  statements  about  your  spatial  and  navigational  abilities, 
preferences,  and  experiences.  After  each  statement,  you  should  circle  a  number  to  indicate  your 
level  of  agreement  with  the  statement.  Circle  “1”  if  you  strongly  agree  that  the  statement  applies 
to  you,  “7”  if  you  strongly  disagree,  or  some  number  in  between  if  your  agreement  is 
intermediate.  Circle  "4"  if  you  neither  agree  nor  disagree. 

1. 1  am  very  good  at  giving  directions. 

strongly  agree  1  2  3  4  5  6  7  strongly  disagree 

2. 1  have  a  poor  memory  for  where  I  left  things, 
strongly  agree  1  2  3  4  5  6  7  strongly  disagree 

3. 1  am  very  good  at  judging  distances, 
strongly  agree  1  2  3  4  5  6  7  strongly  disagree 

4.  My  "sense  of  direction"  is  very  good, 
strongly  agree  1  2  3  4  5  6  7  strongly  disagree 

5. 1  tend  to  think  of  my  environment  in  terms  of  cardinal  directions  (N,  S,  E,  W). 
strongly  agree  1  2  3  4  5  6  7  strongly  disagree 

6. 1  very  easily  get  lost  in  a  new  city. 

strongly  agree  1  2  3  4  5  6  7  strongly  disagree 

7. 1  enjoy  reading  maps. 

strongly  agree  1  2  3  4  5  6  7  strongly  disagree 

8. 1  have  trouble  understanding  directions, 
strongly  agree  1  2  3  4  5  6  7  strongly  disagree 
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9. 1  am  very  good  at  reading  maps. 

strongly  agree  1  2  3  4  5  6  7  strongly  disagree 

10. 1  don't  remember  routes  very  well  while  riding  as  a  passenger  in  a  car. 
strongly  agree  1  2  3  4  5  6  7  strongly  disagree 

1 1. 1  don't  enjoy  giving  directions. 

strongly  agree  1  2  3  4  5  6  7  strongly  disagree 

12.  It's  not  important  to  me  to  know  where  I  am. 
strongly  agree  1  2  3  4  5  6  7  strongly  disagree 

13.1  usually  let  someone  else  do  the  navigational  planning  for  long  trips, 
strongly  agree  1  2  3  4  5  6  7  strongly  disagree 

14. 1  can  usually  remember  a  new  route  after  I  have  traveled  it  only  once, 
strongly  agree  1  2  3  4  5  6  7  strongly  disagree 

15. 1  don't  have  a  very  good  "mental  map"  of  my  environment, 
strongly  agree  1  2  3  4  5  6  7  strongly  disagree 
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Intentionally  lelt  blank. 
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Appendix  D.  Perspective  Taking/Spatial  Orientation  Test 


This  appendix  appears  in  its  original  form,  without  editorial  change. 
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This  is  a  test  of  your  ability  to  imagine  different  perspectives  or  orientations  in  space.  On  each  of 
the  following  pages  you  will  see  a  picture  of  an  array  of  objects  and  an  "arrow  circle"  with  a 
question  about  the  direction  between  some  of  the  objects.  For  the  question  on  each  page,  you 
should  imagine  that  you  are  standing  at  one  object  in  the  array  (which  will  be  named  in  the 
center  of  the  circle)  and  facing  another  object,  named  at  the  top  of  the  circle.  Your  task  is  to  draw 
an  arrow  from  the  center  object  showing  the  direction  to  a  third  object  from  this  facing 
orientation. 

Look  at  the  sample  item  on  the  next  page.  In  this  item  you  are  asked  to  imagine  that  you  are 
standing  at  the  flower,  which  is  named  in  the  center  of  the  circle,  and  facing  the  tree,  which  is 
named  at  the  top  of  the  circle.  Your  task  is  to  draw  an  arrow  pointing  to  the  cat.  In  the  sample 
item  this  arrow  has  been  drawn  for  you.  In  the  test  items,  your  task  is  to  draw  this  arrow.  Can 
you  see  that  if  you  were  at  the  flower  facing  the  tree,  the  cat  would  be  in  this  direction?  Please 
ask  the  experimenter  now  if  you  have  any  questions  about  what  you  are  required  to  do. 

There  are  12  items  in  this  test,  one  on  each  page.  For  each  item,  the  array  of  objects  is  shown  at 
the  top  of  the  page  and  the  arrow  circle  is  shown  at  the  bottom.  Please  do  not  pick  up  or  turn  the 
test  book-  let,  and  do  not  make  any  marks  on  the  maps.  Try  to  mark  the  correct  directions  but  do 
not  spend  too  much  time  on  any  one  question. 

You  will  have  5  minutes  for  this  test. 
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1 .  Imagine  you  are  standing  at  the  car  and  facing  the  traffic  light. 
Point  to  the  stop  sign. 

traffic 

light 
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Figure  D-4.  Perspective  and  Orientation  Test  Number  4. 
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5.  Imagine  you  are  standing  at  the  stop  sign  and  facing  the  tree. 
Point  to  the  traffic  light. 
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6.  Imagine  you  are  standing  at  the  stop  sign  and  facing  the  flower. 
Point  to  the  car. 


flower 


Figure  D-6.  Perspective  and  Orientation  Test  Number  6. 
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Figure  D-7.  Perspective  and  Orientation  Test  Number  7. 
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8.  Imagine  you  are  standing  at  the  house  and  facing  the  flower. 
Point  to  the  stop  sign. 

flower 


Figure  D-8.  Perspective  and  Orientation  Test  Number  8. 
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9.  Imagine  you  are  standing  at  the  car  and  facing  the  stop  sign. 
Point  to  the  tree. 


stop 

sign 


Figure  D-9.  Perspective  and  Orientation  Test  Number  9. 
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Figure  D-10.  Perspective  and  Orientation  Test  Number  10. 
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11.  Imagine  you  are  standing  at  the  tree  and  facing  the  flower. 
Point  to  the  house. 


flower 


Figure  D-l  1.  Perspective  and  Orientation  Test  Number  11. 
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12.  Imagine  you  are  standing  at  the  cat  and  facing  the  house. 
Point  to  the  traffic  light. 

house 


Figure  D-12.  Perspective  and  Orientation  Test  Number  12. 
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Appendix  E.  Final  Questionnaire 


This  appendix  appears  in  its  original  form,  without  editorial  change. 


59 


a 


Final  Questionnaire  -  Navigation 


1.  Use  the  scale  below  to  rate  your  LEVEL  OF  COMFORT  in  each  environment:  ( write  the 
appropriate  number  in  the  box  beneath  the  corresponding  environment): 


Scale 


Very 

Slightly 

Slightly 

Very 

uncomfortable 

Uncomfortable 

Uncomfortable 

Neutral 

Comfortable 

Comfortable 

Comfortable 

1 

2 

3 

4 

5 

6 

7 

Navigating 

around 

Warehouse  with 
no-load 

Navigating 

around 

Warehouse  with 
53-lb  load 

Navigating  using 
Computer 
Monitor  and 
Joystick 

Navigating  in  IES  using 
Joystick 

— 

— 

— 

— 

2.  Use  the  scale  below  and  rate  the  PHYSICAL  EFFORT  REQUIRED  to  perform  the  following 
conditions  ( write  the  appropriate  number  in  the  box  beneath  the  corresponding  environment ): 


Scale 


Very,  Very 
Light 

1 

Very  Light 

2 

Fairly  Light 

3 

Somewhat 

Hard 

4 

Hard 

5 

Very  Hard 

6 

Very,  Very 
Hard 

7 

Navigating 

Around 

Warehouse  with 
No-load 

Navigating 

Around 

Warehouse  With 
53-lb  load 

Navigating  using 
Computer 
Monitor  and 
Joystick 

Navigating  in  IES  using 
Joystick 

— 

— 

— 

— 
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3.  Use  the  scale  below  and  rate  the  how  hard  or  easy  it  was  to  do  the  NAVIGATION  TASKS  in 
each  environment  ( write  the  appropriate  number  in  the  box  beneath  the  corresponding 
environment ): 


Scale 


Very,  Very 
Hard 

Very  Hard 

Hard 

Somewhat 

Easy 

Easy 

Very  easy 

Very,  Very 
Easy 

1 

2 

3 

4 

5 

6 

7 

Navigating 
Around 
Warehouse 
With  No-load 

Navigating 

Around 

Warehouse  With 
53-lb  load 

Navigating  Using 
Computer 
Monitor  and 
Joystick 

Navigating  in  IES  using 
Joystick 

— 

— 

— 

0151 

4.  In  each  pair  of  environments  (across  each  row),  please  circle  the  environment  in  which  it  was 
easier  to  perform  the  navigation  task: 


Warehouse,  no-load 

Warehouse,  with  53  lb.  load 

Computer  workstation  with  joystick 

IES  with  joystick 

Warehouse,  no-load 

Computer  workstation  using  joystick 


Warehouse,  with  53  lb.  load 
Computer  workstation  using  joystick 
IES  with  joystick 
Warehouse,  with  53  lb.  load 
IES  with  joystick 
Warehouse,  no-load 


5.  For  each  of  the  tasks  below,  rate  which  environment  was  easiest  perform  the  navigation  task, 
and  which  was  most  difficult.  Use  the  numbers  1  through  4,  where  1  is  the  easiest  for  navigating, 
2  was  more  difficult  than  1,  3  was  more  difficult  than  2,  and  4  was  the  most  difficult  for 
navigating.  Use  all  numbers  and  put  a  different  number  in  each  box: 


Task 

Rating 

Navigation  task  around 
warehouses  with  no-load 

— 

Navigation  task  around 
warehouses  with  53-lb  load 

— 

Navigation  task  while  using  the 
computer  monitor,  using  a 
joystick 

— 

Navigation  task  while  in  the  IES, 
using  a  joystick 

— 
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b.  Final  Questionnaire  -  Auditory  Monitoring  Task 

1.  Use  the  scale  below  to  rate  your  LEVEL  OF  COMFORT  in  each  environment:  ( write  the 
appropriate  number  in  the  box  beneath  the  corresponding  environment ): 

Scale 


Very 

Uncomfortable 

1 

Uncomfortable 

2 

Slightly 

Uncomfortable 

3 

Neutral 

4 

Slightly 

Comfortable 

5 

Comfortable 

6 

Very 

Comfortable 

7 

Auditory  Task 
Around 
Warehouse 
With  No-load 

Auditory  Task 
Around 

Warehouse  With 
53-lb  load 

Auditory  Task 
using  Computer 
Monitor  and 
Joystick 

Auditory  Task  in  IES 
using  Joystick 

— 

— 

— 

— 

2.  Use  the  scale  below  and  rate  the  PHYSICAL  EFFORT  REQUIRED  to  perform  the  following 
conditions  ( write  the  appropriate  number  in  the  box  beneath  the  corresponding  environment ): 


Scale 


Very,  Very 
Light 

1 

Very  Light 

2 

Fairly  Light 

3 

Somewhat 

Hard 

4 

Hard 

5 

Very  Hard 

6 

Very,  very 
Hard 

7 

Auditory  Task 
around 

Warehouse  with 
No-load 

Auditory  Task 
Around 

Warehouse  With 
53-lb  load 

Auditory  Task 
Using  Computer 
Monitor  and 
Joystick 

Auditory  Task  in  IES 
Using  Joystick 

— 

— 

— 

— 

3.  Use  the  scale  below  and  rate  the  how  hard  or  easy  it  was  to  do  the  AUDITORY  TASK  in  each 
environment  (write  the  appropriate  number  in  the  box  beneath  the  corresponding  environment): 

Scale 


Very,  Very 
Hard 

1 

Very  Hard 

2 

Hard 

3 

Somewhat 

Easy 

4 

Easy 

5 

Very  Easy 

6 

Very,  Very 
Easy 

7 

Auditory  Task 
Around 
Warehouse 
With  No-load 

Auditory  Task 
Around 

Warehouse  With 
53-lb  Load 

Auditory  Task 
Using  Computer 
Monitor  and 
Joystick 

Auditory  Task  In  IES 
Using  Joystick 

— 

— 

— 

— 

4.  In  each  pair  of  environments,  please  circle  the  environment  in  which  it  was  easier  to  perform 
the  auditory  task: 
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Warehouse,  no-load 

Warehouse,  with  53  lb.  load 

Computer  workstation  with  joystick 

IES  with  joystick 

Warehouse,  no-load 

Computer  workstation  using  joystick 


Warehouse,  with  53  lb.  load 
Computer  workstation  using  joystick 
IES  with  joystick 
Warehouse,  with  53  lb.  load 
IES  with  joystick 
Warehouse,  no-load 


5.  For  each  of  the  tasks  below,  rate  which  environment  was  easiest  perform  the  auditory  task,  and 
which  was  most  difficult.  Use  the  numbers  1  through  4,  where  1  is  the  easiest  for  performing  the 
auditory  task,  2  was  more  difficult  than  1,  3  was  more  difficult  than  2,  and  4  was  the  most 
difficult  for  performing  the  auditory  task.  Use  all  numbers  and  put  a  different  number  in  each 
box: 


Task 

Rating 

Auditory  task  around 
warehouses  with  no-load 

— 

Auditory  task  around 
warehouses  with  53-lb  load 

— 

Auditory  task  while  using  the 
computer  monitor,  using  a 
joystick 

— 

Auditory  task  while  in  the  IES, 
using  a  joystick 

— 
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